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AREA-PRESERVING SURFACE DIFFEOMORPHISMS 


ZHIHONG XIA 


Abstract. We prove some generic properties for C T , r = 1,2,..., oo, area¬ 
preserving diffeomorphism on compact surfaces. The main result is that the 
union of the stable (or unstable) manifolds of hyperbolic periodic points are 
dense in the surface. This extends the result of Franks and Le Calvez |SJ on 
S 2 to general surfaces. The proof uses the theory of prime ends and Lefschetz 
fixed point theorem. 


1. Introduction and statement of main results 

Let Diff^(M) be the set of all C r , r = 1, 2,..., oo diffeomorphisms of compact 
orientable surface M that preserves a smooth area element p. A property for C r 
area-preserving diffeomorphisms of M is said to be generic , if there is a residual 
subset R C Diff^(M) such that the property holds for all / £ R. The following are 
some examples of known generic properties: 

G1 All the periodic points are either elliptic or hyperbolic. This is proved by 

Robinson ( 29 - 

G2 All elliptic periodic points are Moser stable. Moser stable means that the 
normal form at the elliptic periodic points are non-degenerate in the sense 
of KAM theory (cf. Siegel & Moser [ 22 ] ). This implies that there are 
invariant curves surrounding each elliptic periodic point. This condition 
requires that the map is sufficiently smooth, say r > 16. 

G3 For any two hyperbolic periodic points p, q , the intersections of the stable 
manifold W s {p) and the unstable manifold W u (q) are transversal. This is 
again proved by Robinson 123 ■ G1 and G3 together are often referred as 
the Kupka-Smale condition for area-preserving diffeomorphisms. 

G4 For any hyperbolic periodic point p, let Ti and T 2 be any two branches of 
the stable manifold or unstable manifold of p, then Ti = T 2 . For S 2 , this 
is proved by Mather m- The general case is proved by Oliveira m 

G5 If M is a two-sphere S 2 or a two torus T 2 , let R be a branch of the 
stable manifold of a hyperbolic periodic point of p and T 2 be a branch of 
the unstable manifold of p, then Ti 0 T 2 ^ 0. For S 2 , this is proved by 
Robinson jJT and Pixton fT7Tt . For T 2 , it is proved by Oliveira jp2|- For 
general surfaces, it is proved by Oliveira m for most homotopy classes of 
maps. For C 1 case and any compact manifold of arbitrary dimension, the 
result is proved by Takens m (see also Xia m for a stronger result). 

Franks & Le Calvez j5| recently showed another remarkable C r generic property 
for area-preserving diffeomorphisms on S 2 . Their results state that the stable and 
unstable manifolds of hyperbolic periodic points are dense in S 2 generically. In 
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this paper, we extend this result to arbitrary orientable surfaces and to arbitrary 
homotopy classes. More precisely, we state our main result. 

Theorem 1.1. Let M be a compact orientable surface. There exists a residual 
subset R C Diff^(M) such that if f £ R and P is the set of all hyperbolic periodic 
points of f, then both the sets U pe pW s (p) and U pe pW u (p) are dense in M. 

Furthermore, let U C M be an open connected subset such that it contains no 
periodic point for f and suppose that, for some hyperbolic periodic point p, U l~l 
W s {p ) yf 0, then W s (p) is dense in U. 

The main tool of our proof is the theory of prime ends a ms and Lefschetz fixed 
point theorem. Also, Arnold’s conjecture for symplectic fixed points, as proved by 
Conley & Zehnder 0 for the case of torus, turned out to be essential for our results 
on T 2 . 

Our work was motivated in our effort to prove the so-called C r closing lemma, 
as conjectured by Poincare m Our result provides a strong evidence supporting 
the C r closing lemma. We will discuss this and other related problems in the end 
of the paper. 

I am grateful to Kamlesh Parwani for useful comments and corrections to the 
original manuscript. 

2. Prime ends and a Theorem of Mather 

Let U be a simply connected domain of S 2 whose complement contains more 
than one point. One can define the prime end compactification of U, introduced 
by Caratheodory g], by adding the circle 5 1 . Each point in S 1 are defined by a 
sequence of nested arcs in U. S 1 inherits a topology from U. We refer to Mather 
m for definitions and general discussions. Here we only summarize what we will 
need in this paper. The compactification U = U U S 1 of U, with inherited topology 
on S 1 , is homeomorphic to a closed disk D. If U C S' 2 is an invariant subset for 
a homeomorphism on S , then the function / extends continuously to a homeo- 
morphism / : U — > U. Moreover, / restricted to the prime ends S 1 is a circle 
homeomorphism. 

When the rotation number p of f\gi is rational, then / has a periodic points on 
the prime ends. But it may happen in general that / does not have any periodic 
points on dU. However, if / is area-preserving, then this will never happen. 

Lemma 2.1. Let f be an area-preserving homeomorphism on S 2 and let U be a 
simply connected, invariant set whose complement contains more than one point. 
Let f : U —> U be the extension of f to the prime end compactification of U. If f 
has a periodic point in its prime ends then f has a periodic point in dU. 

The proof is simple, we refer to Mather m and Franks & Le Calvez J5J. The 
next theorem, which is due to Mather m states that in generic situations, there 
are no periodic points on the prime ends. 

Theorem 2.2. Let f be an area-preserving diffeomorphism on S 2 and let U be a 
simply connected, invariant set whose complement contains more than one point. 
Further assume that f satisfies the generic conditions Gl, G2 and G3. Then there 
is no periodic point for f on the boundary of U and as a consequence, there is no 
periodic point for f on the prime ends and the rotation number of f on the prime 
ends is irrational. 
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The proof is based on the following ideas: by generic conditions G1 and G2, / 
can not have any elliptic periodic points on dU. If there is any hyperbolic periodic 
points on dU , then the boundary of U has to contain some branches of the stable 
and unstable manifolds of the hyperbolic periodic points and this will lead to a 
contradiction to the generic property G3. For details of the proof, see Mather na 
and Franks & Le Calvez 

The theory of prime ends can be easily generalized to other orientable compact 
surfaces M, possibly with boundaries. Let U C M be a connected open domain 
in M. Assume that the boundary of U contains finite number of connected pieces 
and each piece contains more than one point. Then U can be compactified by 
adding prime ends. In this case, the prime ends are homeomorphic to a union of 
finitely many circles S 1 . The number of circles S 1 added is the number of boundary 
pieces seeing from the inside of U. It may be more than the number of connected 
components of dU. The compactification of U , denoted as U, is a compact surface 
with boundaries, each S 1 will be a hole for U. If / : M —> M is a homeomorphism 
and U is invariant for /, then / extends continuously to a homeomorphism on U. 
We denote this extension by /. 

Lemma m and Theorem E2 with obvious modifications, are both true for U. 

In case where U has infinitely many connected boundary components, or where 
some of the boundary components are single points, one can do a prime end com¬ 
pactification to finitely many isolated connected boundary pieces that contain more 
than one point. Again, the number of boundary pieces are defined to be the number 
seeing from inside U 


3. Periodic-point-free regions 

Let R be a residual subset of Diff^(M) such that for any map in R the generic 
properties Gl, G2, G3 and G4 hold. Whenever G4 is assumed, we require the map 
to have sufficient smoothness, say r > 16. Our final result will be true for any 

r = 1,2,...,. 

Fix a map f £ R. Let U C M be a connected open set such that / has no 
periodic point in U. Our goal is to show that either U is filled by stable and 
unstable manifolds of some hyperbolic periodic points, or by an arbitrarily small 
C r perturbation of /, we can create a periodic point in U. 

Let Ajj be the set of all points whose orbit under / passes through U. i.e., 

A u =u£_ 00 f(U). 

Since / is area-preserving, almost every point is recurrent. Since M has finite area, 
Ajj has finite number of connected components. Each of these components are 
open and periodic. Without loss of generality, we may assume that Ajj has only 
one component and this component certainly contains U. The case with more than 
one components can be reduced to this case simply by considering powers of /. 

The set Ajj is invariant under / and it is periodic point free. We are interested 
in whether the closure of Ajj contains any periodic point. The set Ajj typically has 
infinitely many connected boundary components, so we can not apply Lemma, |2. II 
and Theorem o or other results obtained by the prime ends directly. We have 
the following simple lemma. 
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Lemma 3.1. The closure of Ajj, An, is a compact invariant set which contains 
no elliptic periodic point for f. Here we assume f has sufficient smoothness so that 
the generic condition G2 holds. 

Proof: Recall that the map / 6 R satisfies the generic property G2, thus all elliptic 
periodic points are Moser stable. This implies that surrounding each elliptic pe¬ 
riodic point there are infinitely many invariant curves. If Ajj contains an elliptic 
periodic point, then Ajj has to intersect an invariant curve with irrational rotation 
number close to the elliptic point. The set Ajj is invariant, thus it completely covers 
that curve. By Birkhoff fixed point theorem, or more generally the Aubry-Mather 
theory, any neighborhood of the invariant curve contains infinitely many periodic 
orbits. This contradicts to the assumption that An is periodic point free. 

This proves the lemma. 

The closure of Ajj may still contain hyperbolic periodic points. We show that 
this happens only if An contains stable or unstable manifolds of hyperbolic periodic 
points. 

Lemma 3.2. If the closure of An, An, contains a hyperbolic periodic point p, then 
U fl W s {jp) / 0 and U H W u (p) ^ 0. 

Proof. Recall that for / € R, generic property G4 implies that W s (p) = W u {jp), it 
suffices to show that either A v fl W s (p) ^ 0 or A v fl W u (p) ^ 0. 

First, we remark that if M = S 2 or M = T 2 and generic property G5 is assumed, 
then the lemma can be easily. In this case, each branch of the stable and unstable 
manifolds of p intersect and the intersection is transversally, there is a homoclinic 
tangle in the neighborhood of p and this homoclinic tangle divides a small neigh¬ 
borhood of p into infinitely many small rectangles. Choose a small neighborhood 
p such that all of these rectangles have areas smaller than that of U. If p is in the 
closure of An, then, for some integer z, f l (U) intersects one of these small rectan¬ 
gles. By the area preserving property of f, /*([/) can not be totally contained in 
any single rectangle and thus it has to intersect the boundary of the rectangle. This 
implies that /*([/) contains a point on either the stable manifold or the unstable 
manifold of p. 

For the general case, suppose that there is a hyperbolic periodic point p £ M 
such that An l~l W s (p) = 0, we want to show that p is not in the closure of An- Let 
V be the connected component of M\W s (p) containing An- Since the complement 
of V is connected, the boundary of V consists of finitely many connected pieces and 
each piece contains more than one point. Therefore V has a prime end extension, V 
with the extended map / on V. By Mather’s theorem m, which is proved in this 
general setting, V has no periodic points on the boundary of V. This implies that 
p V and since An C V, this implies that the closure of An contains no periodic 
point. 

This proves the lemma. 

To get rid of isolated points in the boundary of An, we let A = ini(An). Then 
any isolated connected boundary piece contains more than one point. If the closure 
of An is periodic point free, then so is the closure of A. 

4. Isotopic to identity cases 

In this section we prove our main lemma in the case where the map is isotopic 
to identity. 
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Lemma 4.1. Let f £ R be a generic diffeomorphism of M and f is isotopic to 
identity. Let A C M be an open, connected, periodic point free, f -invariant set 
whose closure contains no periodic point. We further assume that A = intA. Then 
either A = M = T 2 or A is homeomorphic to an open annulus and its prime end 
extension is a closed annulus. 


Proof. We first assume that M 7 ^ T 2 . Since the Euler characteristic of M is non¬ 
zero, by Lefsclretz fixed point theorem, / contains at least one fixed point. This 
implies that A A Af. Let Fix(/) be the set of all fixed point for /. It contains 
finite number of points and it is contained in the interior of M\A. Let B C M 
be the union of the connected components of M\A intersecting Fix(f). The set 
B is closed and has finite number of connected components and each contains at 
least one fixed point of f. Let C = M\B , obviously C is open, A C C and the 
closure of C contains no fixed point. Let C be the prime end extension of C and 
let / : C —► C be the extension of f. Then / has no fixed point. The set C is a 
compact surface with boundary, its topology is uniquely determined by the number 
of handles and the number of holes on S 2 . Let the number of handles of C be 
k and the number of holes be l. The map / keeps invariant of all the boundary 
pieces. The Euler characteristic number of C is 2 — 2k — l. Even though / may not 
neccessarily be homotopic to identity, but its induced map on homology is identity. 
Hence its Lefsclietz number is the same as its Euler characteristic number. Since 
/ has no fixed point, this implies that 2 — 2k — l = 0. This implies that either 
k = 0, l = 2 or k = 1, l = 0. The latter implies that C is a torus and thus M is a 
torus, which we assumed was not the case. The first case implies that C is a closed 
annulus. 

We claim that A = C. If not, there is a point x £ C\A. i.e., x is in the interior 
of C\A For if no such point exists, then C C A. But C is open, this implies that 
C C intA = A, by the assumption on A. Let B x be the connected component in 
C\A containing x. Then B x is closed and B x fl dC = 0. For if B x n dC A 0> then 
B x C B, where B is the set of connected components containing fixed points in 
the complement of A, as defined previously. Since B x has positive area, it must 
be periodic under /. Let k be the period of B x . Let Fix(/ fc ) be the set of all 
periodic points of / with period k. Again, Fix(/ fc ) contains only finite number of 
points. Let Bk be the union of all connected components of C\A. The set Bk has 
finite number of, say l, connected components and all components are periodic with 
period k. Moreover, B fl dC = 0. Let C x = C\Bk, then C x is an open set and 
C x = int C x . The prime end extension of C x is a closed annulus with l interior disks 
removed, and there is no periodic points of period k in C x . However, the Lefschetz 
number for f k on C x is l A Ck this implies that f k has to have at least one fixed 
point in C x . This contradiction proves that A = C. i.e., A is an annulus. 

If M = T 2 , then either A = T 2 or there is a point x £ int(M\A). Let B' be 
the connected components containing x in the complement of A. Again, B' must 
be periodic and let this period be k. Let B be the union of all the connected 
components of M\A that are fixed under f k . 0 A B' C B. Let C = A\B. Then 
this case follows from above arguments by considering f k on C. 

This proves our lemma. 
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5. A LEMMA ON PERIODIC POINTS 

Now we consider the case where / is not necessarily isotopic to identity. To prove 
the same result, we need to prove a simple theorem on the existence of periodic 
points for maps of compact surfaces with boundaries. 

Lemma 5.1. Let M be a compact, connected surface, possibly with boundary. As¬ 
sume that M has nonzero Euler characteristic. Suppose that f : M —► M is a 
homeomorphism, then f has a periodic point. Moreover, for any positive integer 
n, there exists infinitely many positive integers i such that the Lefschetz number 
L(f nl ) is negative. 

Proof. Let L(f) be the Lefschetz number of the map /. By Lefschetz fixed point 
theorem, it suffices to show that L(f n ) is nonzero for some nonzero integer n. 

We consider homology groups of M with real coefficients. If H' 2 (M) 7 ^ 0, by 
taking an interate of /, we may assume that the induced map on H 2 (M) is the 
identity. Let tr\ (/*) be the trace of the induced map /» : Hi(M) —> 
then L(f n ) = 2 — iri(/") if M is orientable and without boundary, and L(f n ) = 
1 — tri(ff’) if M is non-orientable or has boundary. The theorem follows easily 
for the cases where the dimension of H±(M) is zero (the sphere and the projective 
plane). The torus, the Klein bottle, the Mobius strip and the annulus all have Euler 
characteristic zero and hence are excluded. The only cases we need to consider are 
where the dimension of the first homology H^M) are greater than or equal to 3. 
Let Ai, A 2 (...) Aj be the eigenvalues of the induced isomorphism /* : Hi(M) —> 
where l is the dimension of H±(M). We can write A= lyct;, where rj is a 
positive real number and at is complex number on the unit circle, for i = 1,2 
There exists a sequence of integers such that —> 1 as k —> 00 , for all 

* = 1 , 2 ,...,/. 

If rj > 1 for some i = 1,2,...,/, then tri(f™ k ) = K k ► 00 as k —> 00 . 
This implies that for large k, L(f nk ) 7 ^ 0, the lemma follows. If fj < 1 for some 
i = 1,2,...,/, then we consider / -1 , the inverse of /. The same argument shows 
that L(f n ) 7 ^ 0 for some negative integer n. Since L(/") = L(f~ n ), the lemma 
again follows. Finally, if r* = 1 for all * = 1,2,...,/, then iri(/" fe ) = Xa=i K k Z 
as k —> 00 . As j > 3, 2 — tri(fff k ) ^ 0 and 1 — ir 1 (/" fc ) 7 ^ 0 for large k. This 
shows that for infinitely many positive integer k, the Lefschetz number L(f nk ) is 
negative. Lefschetz fixed point theorem concludes that there are at least one fixed 
point for f nk for such k. 

For any positive integer n , replacing / with f n , the above arguments show that 
there are infinitely many positve integer i such that L(f nl ) is negative. 

This proves the lemma. 

For orientable compact surfaces, only T 2 and annulus have zero Euler charac¬ 
teristics. Lemma rm would follow from the above lemma if the periodic point free 
set A in Lemma l4.1l has a prime end extension that makes it into a compact mani¬ 
fold with boundary. We have to show that A can’t have infinitely many boundary 
pieces. The proof of the lemma is basically a verification of this fact. 

6. General case 

Let f € R be a generic diffeomorpliism on the compact surface M. Let A C M 
be an open, connected, periodic point free, /-invariant set whose closure contains no 
periodic point. Assume A = int(A). For any integer i, we let Fix(/*) be the set of 
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all periodic points of / with period i. If A M, M\A is nonempty and contains at 
least one connected component. The number of the connected components in M\A 
may be infinite. Let Bi be the set of all connected components of M\A containing 
a periodic point of period i. Bi is closed and Bi n Fix(/ 1 ) / 0, if Fix(/ 1 ) ^ 0. 
Bi has finite number of connected components. For any positive integers i.j. each 
component of Bj is either a component of Bi or disjoint from Bi. If j = ki for some 
positive integer k, then Bi C Bj. 

Let Ci = M\Bi. Then Ci has finite number of boundary components. Let Ci 
be the prime end extension of Ci and it is a compact surface with boundary. Let 
the number of handles of Ci be m and the number of holes be n. For any integer k, 
the number of handles of Cki is smaller than or equal to m. Therefore, there exists 
an integer ** such that the number of handles of Cki * is a constant for all positive 
integer k. 

If Fix(/*) is empty for all positive integer i, then either A = M, then we set 
Ci = M for all i, or A ^ M, then we pick a point x £ M\A and let Bi be the 
connected component of M\A and its iterates under /. Since / is area preserving, 
Bi has finite number of components. 

By our construction, /' : Ci —» Ci has no fixed points for all i > 1. 

We are ready to prove Lemma F-hll tor the general case. 

Lemma 6.1. Let f £ R be a generic diffeomorphism of M. Let A C M be an open, 
connected, periodic point free, f -invariant set whose closure contains no periodic 
point. We further assume that A = intA. Then either A = M = T 2 or A is 
homeomorphic to an open annulus and its prime end extension is a closed annulus. 


Proof: First if there exists a positive integer k such that f k is isotopic to identity, 
the lemma is proved in the same way as Lemma 14. II One just need to consider f k . 

We first suppose that M/T 2 . 

As described before, there exists an integer i* such that the number of handles 
of Cki * is a constant for all positive integer k. We now consider f l : Ci• —> Ci* . It 
is a homeomorphism on compact surface, keeping invariant of all boundary pieces. 
Suppose f l : Ci* —> Ci* has no periodic points, then by Lemma o Ci* is an 
open annulus and Ci* is a closed annulus. We claim that A = Ci*. Suppose not, 
then there is a point x £ Ci* , but x £ A. Since A = intA and by the definition of 
Ci*, x dA. Let C x be the connected component of C;*\A containing x. Then 
C x an open disk in a annulus. Since / is area preserving, C x must be periodic and 
it contains a periodic point. This contradicts to the assumption that Ci* has no 
periodic point. 

Now suppose that f l : Ci* —> Ci* has a periodic point x with period p. By the 
assumption on A, x ^ A. By Lemma l5~Tl there is a positive integer j such that the 
Lefschetz number of the pj’s iterate of f 1 on Ci * is negative, i.e., L((/**) w \Ci*) < 
0, where p is the period of x. Now consider the prime end extension of Ci* P j. By 
our choice of i* , Ci* P j is topologically Ci* with finitely many, say k, k > 0 , open 
disks removed and these disks are periodic with period pj under the extended map 
f l . We have the following relations on the Lefschetz numbers L(f l w \C z * p j) = 
L((f l r\Ci*) — k < 0. Lefschetz fixed point theorem implies that f l PJ has a fixed 
point on Ci* p j, which is impossible by the definition of Ci* P j. This contradiction 
shows that /' : C,» —► Ci* has no periodic point and hence A = Ci* is an annulus. 
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We are left with one case: M = T 2 and A ^ M. The same argument works in 
this case too. 

This proves the lemma 


7. Maps on annulus 

Let U be a connected open subset of M and let Au = U Z-o.PiU). If the 
closure of Au contains no periodic point, then A = int(A[/) is open, containing no 
periodic points in its closure. By the above lemma, if M ^ T 2 , then A is a union of 
finite disjoint open annuli, periodic under /. The dynamics on Annulus have been 
well studied (cf. Franks 0, Le Calvez & Yoccoz ]3] The following lemma shows 
that, if A is an annulus, we can perturb / with an arbitrarily small C r perturbation 
to create a periodic point in U. The same result was also used in 

Lemma 7.1. Fix f £ R and assume M ^ T 2 . Let U be a connected open subset of 
S 2 and Au = U fZ_ 00 f l {U). Assume that U does not intersect any stable or unstable 
manifolds of hyperbolic periodic points of f. Then for any C r neighborhood V of f, 
there exists g £ V such that the support of g — f is contained in the interior of the 
closure of Au and g has a periodic point in U. 

Proof. Since U does not intersect any stable or unstable manifolds of hyperbolic 
periodic points of /, By above lemmas, / has no periodic point in Au and no 
periodic point in the closure of Au and therefore, A = int(A(/) is a union of finite 
disjoint periodic annuli. Without loss of generality, we may assume that Au itself 
is an open annulus. 

Using prime end extension, we obtain an area-preserving continuous map on the 
prime end closure of Au, still denoted by Au- Let (x, y), x £ 1 ( mod 1), y £ [0,1] 
be a coordinate on Au- Since / preserves invariant measure p, by Birkhoff Ergodic 
Theorem, for /Lt-almost every point z = (x, y) £ Au, the rotation number 

P\Z, /) = lim -:- 

i —»oo l 

is well defined. Here ir x is the projection on Au into its first coordinate and / is 
a lift of / to its universal cover lx [0,1]. A different lift of / yields a different 
rotation number that differs by an integer. 

Since there is no periodic points in Au, by Franks theorem |^j, there exists an 
irrational number a £ R such that for almost all 2 £ Au, the rotation number 
p(z, f) exist and p(z, f) = a. In particular, if z is in the boundaries of Au then 
P(z,f) = a. 

Let 7 be a simple closed curve in the interior of Au■ We may assume that 7 is 
homotopically non-trivial in Au- Take a small tubular neighborhood 7 $ of 7 in the 
interior of Au and parametrize this tubular neighborhood by 7 $ : S 1 x [—<5, 5] —►: Au 
for some small 6. In fact, for convenience we may even assume that 75 is area¬ 
preserving. Let P : [—<5, <5] —> R be a C°° function such that fi(t) > 0 for all 
—5<t<5 and (3{—S) = (3 (<5) = 0 and /3 is C°° flat at ±<5. i.e., all the derivatives 
of P(t ) at ±<5 are zero. 

Let h t : Au —> Au be a C°° diffeomorphism such that if z £ 75 , h t {z) = z 
and if z £ 7 s, h e = js 0 T e o ( 7 a ) -1 where T e (6,t) = (6 + e/?(t),i) for all 9 £ S 1 
and t £ [—<5, <5]. We remark that h e —> Id in C°° topology as e —» 0 and the mean 
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rotation number for h e with respect to the area p, is 

p - iAu ' h - ,= juk)L emdt 

Therefore for any e > 0, p^(Au,h e o f) = p^{Au,h e ) + p^(Au,f) > a, this 
implies that there exists a point y* £ Au , such that p(y*,h e of) > a. Since 
p(z, h e o /) = p(z, f) = a for all z £ dAjj , we conclude, from Franks’ theorem 0, 
that for any rational number p/q, such that a < p/q < p(y*, h t o /), there exists a 
periodic point of period p/q for the map h e o /. 

Thus, for any e > 0, there are infinitely many periodic points for h e o / in the 
interior of Au. In fact, all of these periodic points have to pass through the strip 
7 s- However, these periodic points may be far away from U. To find periodic points 
in U, we need to do some estimates on these orbits. 

Since Au = Uiez/*(^), for any point z £ 75 C Au, there exists an integer n z £ Z 
and a neighborhood of 2 , W z C Au, such that f Hz {W z ) C U. {W z , z £ 75 } forms 
an open cover for the compact set 7 5 . Let W Z 1 ,W Z2 , ..., W Zk be a finite subcover 
of 7 s and let N = max{|n zi |, |n Z2 |,..., |n z J}. 

The integer N chosen above has a very important property: for any z £ 75 , 
the orbit segment {f~ N (z),f~ N+ 1 (z),...,f N {z)} intersects U at least once. Or 
equivalently, the set U^L_ N f l (U) covers 7 g. Since U is open, this same prop¬ 
erty holds for all g sufficiently close to / in C° topology, i.e., the orbit segment 
{g~ N (z), g~ N+ 1 (z),... ,g N (z)} intersects U for all 2 £ 7 s, provided that g is suffi¬ 
ciently close to /. 

Above arguments show that if e > 0 is small enough, h e o / has infinitely many 
periodic orbits and all of these periodic orbits intersect U. 

This proves the lemma. 

8. Maps on torus and Arnold’s conjecture 

The final case is where M = T 2 and / has no periodic point. We will show that 
such / is not generic and it can be perturbed to create a periodic point. 

Let /* 1 : LZi(T 2 ,R) = R 2 —> R 2 be the induced map on the first homology of 
T 2 . Let Ai, A 2 be the eigenvalues of /* 1 , Ai = A 2 . Since / has no periodic point, 
the Lefschetz number L(f k ) = 0 for all k. This implies that X k + X k =2, for all k. 
We must have Ai = A 2 = 1. Since /* 1 and its inverse are both integer matrices, we 
have only two choices: / is isotopic to identity, where /* 1 = I or / is isotopic to a 
Dehn twist, i.e., for some integer k =£ 0, 



We first consider the case where / is isotopic to identity. The proof is basically 
an application of the Arnold conjecture [Q as proved by Conley and Zehnder (5j. 

Lemma 8.1. Let f : T 2 —> T 2 be an area-preserving diffeomorphism such that f is 
isotopic to identity. Then for any C r neighborhood V of f, there exists g £ V such 
that g : T 2 —► T 2 has a periodic point. 

Proof. 

Let / : M 2 —> M 2 be a lift of the map / on T 2 = M 2 /Z 2 to its universal cover 
R 2 . Let 77 , i = 1,2 be the projection of R 2 to its first and second coordinates 



10 


ZHIHONG XIA 


respectively. The average rotation numbers for the map /, are defined to be 

Pi ( f ) = j (7Ti(/(p)) - ^ i { p )) dp . 

The combination p(f) = (pi{f), p 2 {f)), m od Z 2 is called the average (or mean) 
rotation vector for /. The rotation vectors are well defined for maps isotopic to 
identity. 

We want to do a small perturbation to / so that each component of the average 
rotation vector is a rational number. This is easy: one composes the map / with 
T (ei,e 2 )( X ,V) = {x 1 +e 1 ,x 2 + e 2 ), then p(f ■ T (ei>e2) ) = p{f) + (e i,e 2 ), mod Z 2 . By 
properly choosing small ei and £ 2 , we obtain a rational rotation vector for /■T( ei e2 ). 

There is a positive number i such that the mean rotation vector for (/■TV ei)e2 \)* fc 
is an integer vector, which is equivalent to zero on the torus. (/■T( ei £2 ))* fc is isotopic 
to identity and preserves a smooth area element. The Arnold conjecture, as proved 
by Conley and Zehnder [jJJ in the case of torus, implies that (/ • T^ l t2 ^) lk has at 
least three fixed points, four if all non-degenerate. This implies that / • T( ei>€2 ) has 
periodic points of period ki. 

This proves the lemma. 

We use Poincare-Birkhoff Theorem for the case where / is isotopic to a Dehn 
twist. 

Lemma 8.2. Let f : T 2 —> T 2 be an area-preserving diffeomorphism such that f is 
isotopic to a Dehn twist. Then for any C r neighborhood V of f, there exists g £ V 
such that g : T 2 —> T 2 has infinitely many periodic points. 

Let ((x,y), mod Z 2 ) be a coordinate systems on T 2 such that / is isotopic to 
the map (x,y) 1 —► ((x + ky,y), mod Z 2 ) for some non-zero integer k. We may 
assume, without loss of generality, that k > 0. In this homotopy class, there are 
maps without any periodic point. For example, the map (a :,y) 1 —> (x + ky,y + a) 
has no periodic point if a is irrational. However, there are periodic points when a 
is rational. Our first step is to perturb the map so that it has a rational vertical 
rotation number. 

Lift the map / in the y direction, we obtain a map on the infinite cylinder 
f y : S 1 x R —> S 1 x R. Define the mean vertical rotation number 

P 2 (f v ) = J (tt 2 (f y (p)) - 7 T 2 {p))dp, 

where p is the area element. ^ 2 {f v {p)) — ^ 2 (p) is independent of choices of the 
covering points. We define the mean vertical rotation number of / to be p 2 (f) = 
P 2 (f y ), mod 1. This is independent of the lift. 

By composing / with the map (x, y) <—> (x,y + e) for some small e, we obtain a 
map g on the torus such that its mean vertical rotation number is rational. This 
implies that there is a positive integer l such that the vertical rotation number of g l 
is zero. Now, choose a lift of g l , G v , in the y direction such that its mean vertical 
rotation number is zero (instead of being a non-zero integer). Then G v is an area 
preserving map on the infinite cylynder S 1 x R which is also exact, i.e., integral of 
the 1 -form ydx over any closed curve on the cylinder is invariant under the map. 
Moreover, if we let G be the lift of G v to R 2 , we have that 7Ti {G(p)) — x — kly is 
uniformly bounded for any p = (x, y) gR 2 . 
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Poincare-BirkhofF twist map theorem implies that there are at least two fixed 
points and infinitely many periodic points for G v . This implies that there are 
infinitely many periodic points for g on T 2 . 

This proves the lemma. 

The Poincare-BirkhofF twist map theorem (or Poincare’s last geometric theorem) 
shows the existence of fixed points for area-preserving maps on the annulus with 
twist condition. Here we have an infinite cylinder with infinite twists on two ends. 
There are two ways to work around this. One is to directly apply Birkhoff’s proof 
[21 to the annulus {\y\ < M } with large M > 0. The annulus is not invariant 
but the proof works in the same way, as long as one has exactness in the area 
preserving property. Another way to prove the result is to modify the map so 
that all horizontal lines are fixed for large values of \y\. Again, the exactness is 
neccessary. Since these techniques are well known, we will not give details here. 

9. Proof of the main theorem 

Let M be a compact surface and let R C Diff[)(M) be the set of area-preserving 
diffeomorphisms on M satisfying G1-G4. We first assume that r > 16. In addition, 
we assume that R satisfies the following two conditions: for any f £ R, 

G6 every invariant open annulus contains a periodic point; 

G7 if M = T 2 , then there is a periodic point. 

By Lemma IQ G7 is an open and dense condition, hence generic. By Lemma 
o G6 is a generic condition. Therefore, R is a residual set. We claim that for any 
f £ R, the stable and unstable manifolds of hyperbolic periodic points are dense 
in M. Suppose this is not true and there exists an open set U C M such that U 
does not intersect stable manifold and unstable manifold of any hyperbolic periodic 
point. Then neither does the invariant set Ajj = Ui e z/ l (f7). Then by Lemma f3.II 
and Lemma E21 the closure of Ajj contains no periodic points. Let A = int(Ap-), 
then the closure of A contains no periodic point. Lemma l(i.II shows that A must be 
an open annulus or M must be a torus. This contradicts to conditions G6 and G7. 

This proves the first part of our theorem for r > 16. For lower smoothness, 
i.e., for r = 1, 2,..., 15, we first note that the residual set R C Diff[)(M), r > 16 
constructed above is dense in Diff)) (M) with r = 1, 2,..., 15. Moreover, for each 
open set U C M, if U intersects a piece of stable (or unstable) manifold of a 
hyperbolic fixed point for some / £ Diff r (M), then same is true for any map 
close to /. Since M has a countable basis of open sets, there is a residual subset 
R r C Diff r AM) for all r = 1,2,..., such that every open set intersects a piece of 
stable (and unstable) manifold of some hyperbolic periodic point. 

This proves the first part of our main theorem. For the second part of the 
theorem, we need the following lemma. 

Lemma 9.1. Every prime end is an accumulation point of periodic points for 
a generic C r surface diffeomorphism. More precisely, there is a residual subset 
R C Diff))(M) (This set can be chosen to be the same as above) such that for 
any f £ R, we have the following property: Let V be an open, f invariant set V 
with finite number of connected boundary pieces and each boundary piece containing 
more than one point, let V be the prime end extension of U, let z £ V be a prime 
end, then there is a sequence of periodic points of f, {pn}jfLi such that p n z as 


n 


oo. 
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We will not give a detailed proof of this lemma. The proof follows from Corollary 
8.9 in Franks & Le Calvez [H], which uses Conley index in a small neigborhood of the 
prime end circle to obtain periodic points (cf. Franks |7j and Le Calvez & Yoccoz 
0). We remark that even though their results are on S 2 , there is no difference 
nearby one piece of prime ends. We also remark that each prime end circle has 
irrational rotation, if a sequence of periodic points approaches the prme end circle, 
then the sequence of periodic orbits approaches every point in the prime end circle. 

Now let f £ R and U be an open connected that contains no periodic point for 
/. Assume that W s (p) fl U / 0 for some hyperbolic periodic point p. Suppose that 
U is not contained in the closure of W s (p), we will derive a contradiction. Let V 
be a connected component of M\W s (p ) whose intersection with U is non-empty. 
Such V exists by our assumption. Let V be the prime end extension of V, then U, 
as a subset of V, is an open neighborhood of an arc in the prime ends. Since the 
rotation number on the prime end circle is irrational, this implies, by the the above 
lemma, U contains infinitely many priodic points. But U is periodic point free, by 
our assumption, this contraction shows that U C W s (p). 

This proves our main theorem. 

10. Other problems and conjectures of Poincare 

Poincare already noted the importance of the generic properties area-preserving 
diffeomorphisms in his study of the three-body problem. The following two funda¬ 
mental conjectures are due to Poincare m 

Conjecture 1. For generic C r area-preserving diffeomorphisms on compact sur¬ 
face M, the set of all periodic points are dense. 

Conjecture 2. There exist a residual set R £ Diffjj(M) such that if f £ R and p 
is a hyperbolic periodic point of f then the homoclinic points of p is dense in both 
stable and unstable manifolds of p. In other words, let J be a segment in W s (p) 
(or W u (p)), then W s (p) fl W u (p) fl J = J. 

In C l topology, both of the above conjectures are proved to be true. The first 
one is a consequence of the so-called closing lemma. It is proved by Pugh m and 
later improved to various cases by Pugh & Robinson m A different proof was 
given by Liao m and Mai m- The second conjecture in C 1 topology is a result 
of Takens ESI - The high dimensional analog was proved by Xia ESI- It can also 
be regarded as a so-called C 1 connection lemma, first proved by Hayashi m and 
later simplified and generalized by Xia |23|, Wen & Xia El. E3- 

In C r topology with r > 1, little progress has been made for these two conjec¬ 
tures and it’s known to be an extremely difficult problem (cf Smale). The local 
perturbation methods used in the C 1 case no longer seem to work and examples 
suggests that a more global approach has to be developed (Gutierrez El)- 

While unable to prove these two conjectures, our main result offers a very strong 
evidence supporting the conjectures. Moreover, as an easy consequence, our result 
implies that the conjecture two implies conjecture one. 
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